\documentclass[12pt]{article}
\usepackage{amsmath}

% Somewhat wider and taller page than in art12.sty
%\topmargin -0.4in  \headsep 0.0in  \textheight 9.0in
%\oddsidemargin 0.25in  \evensidemargin 0.25in  \textwidth 6.5in

\footnotesep 14pt
\floatsep 28pt plus 2pt minus 4pt      % Nominal is double what is in art12.sty
\textfloatsep 40pt plus 2pt minus 4pt
\intextsep 28pt plus 4pt minus 4pt

\topmargin -1cm
\headsep 0mm
\oddsidemargin 1mm
\evensidemargin 1mm
\textwidth 162mm
%\textheight 21cm
\textheight 9.5in
\begin{document}
\begin{center}
\fbox{Ensemble format and jackknife prescription}
\vspace{2mm}

18 November, 2006
\end{center}

\section{Ensemble format}

Ensemble files are in ascii format containing at least one ``quantum number''.
The format is
\begin{verbatim}
<#correlators> <time extent> <type = 0,1> <spatial extent> <number of columns>
0 <real or complex number at slice 0>  [... <real or complex number>]   // config 1
1 <real or complex number at slice 1>  [... <real or complex number>]
...
N-1 <real or complex number at slice N-1>  [.... ]
0 <real or complex number at slice 0>  [... <real or complex number>]   // config 2
1 <real or complex number at slice 1>  [... <real or complex number>]
...
N-1 <real or complex number at slice N-1>  [.... ]
...
\end{verbatim}
The {\rm type=0} designates {\rm real} and {\rm type=1} is {\rm complex}.

For example, a single complex observable with time extent 2, three correlators
and spatial extent 4 might look like
\begin{verbatim}
3 2 1 4 1
0  5.18 3.5
1  1.2  8.0
0  5.17 3.49
1  1.21 8.1
0  5.19 3.51
1  1.21 8.05
\end{verbatim}

\section{Jackknife}

Given some statistical sample $y_i$ the mean and variance are
\begin{eqnarray}
\bar{y} &=& \frac{1}{N}\sum_i^N y_i\\
{\rm var}(y) &=& \frac{1}{N}\sum_i^N \left(y_i - \bar{y}\right)^2\label{var}
\end{eqnarray}
The jackknife prescription for error propagation states that one first
takes the observable $y_i$ in {\em raw} (also called {\em ensemble})
data format and convert it {\em jackknife} format via
\begin{eqnarray}
y_i' &=& \frac{1}{N-1}\sum_{j\ne i}^N y_j \\
     &=& \bar{y} + \left(y_i - \bar{y}\right) \frac{1}{f} \label{jack}
\end{eqnarray}
where $f=-(N-1)$. One sees that the fluctuations of the observable
are scaled down by the factor $f$. One then uses the observables in some
linear or nonlinear function, and then rescales the fluctuations up to raw
format. So, one does
\begin{eqnarray}
z_i' &=& g(y_i',x_i',...)\label{nonlinear}\\
z_i  &=& \bar{z} + \left(z_i' - \bar{z}\right) f \quad.
\end{eqnarray}
The sample variance of $z$ is computed via the conventional method given 
by Eq.~\ref{var}.

\section{Bootstrap}

The bootstrap prescription is similar to jackknife in Eq.~\ref{jack},
except now the rescaling factor $f=\sqrt{N-1}$, and the selection of
the bins $i$ are chosen randomly out of the set of $N$ samples. The
total length of the bootstrap sample (say $M$) can in factor be
greater than $N$.  Otherwise, the operation sequence of rescaling of
the fluctuations down, nonlinear operation, rescaling the fluctuations
up are the same as in jackknife.

\section{Ensbc}

The function $g$ in Eq.~\ref{nonlinear} above can be quite involved
(maybe one's entire code except for the variances). However, the
jackknife sequence of operations can be broken down into basic
operations like, {\tt operator+}, {\tt operator-}, {\tt operator*},
{\tt operator/}, {\tt log}, {\tt sqrt}, etc. The package {\tt ensbc}
available from {\tt http://www.usqcd.org/usqcd-docs/ensbc} provides
a binary calculator that works on files in {\em ensemble} format. 

For example, say there are compatible ensemble files named ``a'', ``b'',
and ``c''. Then, one can do
\begin{verbatim}
% ensbc
d = a + b*c^3
e = b / c
f = sqrt(d*a) / log(e)
calc(f)
\end{verbatim}
%
The files ``d'', ``e'', and ``f'' will be written to disk in ensemble format
with the means and variance propagated via the jackknife prescription in
the individual operations like {\tt operator*}, etc. The parser for the
binary calculator is fully general - any complicated expression that can
be written on a single line can be used. The results of ``calc(f)'' will print
the time slice, the mean, the standard error, and for convenience the error/mean
(noise to signal ratio).

\section{Adat}

The calculator {\tt ensbc} is file based and is most suitable for
simple operations. The package {\tt adat} (located at {\tt
http://www.usqcd.org/usqcd-docs/adat}) has several modules, one of
which supports the jackknife operations described above in a C++
operator overloaded environment.  Namely, one can declare variables as
an ensemble type, with subsequent operations done in jackknife. An example is
as follows.
\begin{verbatim}
#include <ensem.h>
using namespace ENSEM;

EnsemReal a,b;
read("filea", a);   // read file on disk
b = sqrt(a) + log(a);
cout << "mean = " << mean(b) << "  error = " << sqrt(variance(b)) << endl;
write("fileb", b);  // write back to disk
\end{verbatim}



\end{document}
